「免費午餐」的終結
數十年來，開發者享受著「順序運算天花板」——一個以 丹納德縮放定律 為每一代新晶片帶來更快時鐘速度的時代。但我們已觸及 功耗牆。效能不再取決於頻率；而是取決於 併發性。要向前推進，我們必須運用 計算思維 來彌合抽象 數值方法 與現代 平行執行模型之間的差距。
精確度與效能的張力
將一個 領域問題 （例如分子動力學）從 多核心主機 轉移到 CUDA 裝置 不僅是語法上的改變；更是一種 問題分解的轉變。當我們進行平行化時，經常會改變運算順序。由於浮點數運算不具結合性，我們面臨一種權衡： 浮點數精確度與準確度。平行運算結果可能在數學上正確，但在數值上與其順序版本產生偏差。
main.py
TERMINALbash — 80x24
> Ready. Click "Run" to execute.
>
QUESTION 1
What is the primary reason the 'Sequential Ceiling' was reached?
The end of Moore's Law entirely.
Thermal limits and the Power Wall hindering frequency scaling.
Lack of developer interest in C++.
The transition to quantum computing.
✅ Correct!
Dennard scaling failed because we could no longer reduce voltage as transistors got smaller, leading to unsustainable heat at high frequencies.❌ Incorrect
Transistor counts still grow, but we can't run them faster sequentially due to the 'Power Wall'.QUESTION 2
According to Amdahl's Law, if 5% of a program is strictly sequential, what is the maximum theoretical speedup?
Infinite speedup.
Approximately 20x.
5x.
100x.
✅ Correct!
1 / (0.05 + 0) = 20. The 5% sequential fraction limits the speedup regardless of parallel resources.❌ Incorrect
The sequential fraction (s) dictates the ceiling: 1/s.QUESTION 3
Why might a parallel Molecular Dynamics simulation yield slightly different results than a sequential one?
The CPU uses 64-bit while the GPU only uses 8-bit.
Floating-point addition is non-associative in parallel execution.
Parallel threads randomly skip calculations.
The CUDA compiler ignores numerical methods.
✅ Correct!
In parallel, operations are often reordered. (A+B)+C != A+(B+C) in floating-point due to rounding error accumulation.❌ Incorrect
Floating-point non-associativity is the culprit; the order of summation matters for precision.QUESTION 4
What does 'Problem Decomposition' involve in the context of parallel programming?
Breaking code into functions for readability.
Mapping domain-specific data to parallel execution models like threads or grids.
Deleting unnecessary variables to save memory.
Compiling the code for multiple OS targets.
✅ Correct!
Decomposition involves identifying how data (data-level) or functions (task-level) can be computed independently.❌ Incorrect
It refers to the structural strategy used to unlock concurrency.QUESTION 5
Which of the following describes the 'Computational Thinking' bridge?
A hardware component between the CPU and GPU.
A framework to translate domain knowledge into architecture-aware algorithms.
An automated AI tool that writes CUDA kernels.
The process of upgrading RAM on a host machine.
✅ Correct!
It is the mental framework that synthesizes domain problems with hardware constraints and programming models.❌ Incorrect
It is a strategic methodology, not a physical component.Case Study: Scaling a Fluid Dynamics Solver
Numerical Consistency in Parallel Architectures
A research team is porting a Navier-Stokes fluid solver from a single-core CPU to a cluster of CUDA devices. While the GPU version is 50x faster, the pressure values at the 100th iteration differ from the sequential version by 0.001%.
Q
1. Is this divergence likely caused by a hardware bug or a fundamental property of parallel computing?
Solution:
It is a fundamental property. In parallel reductions, the order of summation for grid cells changes. Due to floating-point non-associativity, these small rounding differences accumulate over time.
It is a fundamental property. In parallel reductions, the order of summation for grid cells changes. Due to floating-point non-associativity, these small rounding differences accumulate over time.
Q
2. How does the 'Sequential Ceiling' explain why the team chose CUDA over a faster single-core CPU?
Solution:
Dennard Scaling has ended; single-core frequency no longer increases significantly. Massive parallelism on CUDA devices is the only way to achieve the required throughput, even if it requires managing numerical accuracy trade-offs.
Dennard Scaling has ended; single-core frequency no longer increases significantly. Massive parallelism on CUDA devices is the only way to achieve the required throughput, even if it requires managing numerical accuracy trade-offs.